Section: New Results

Numerical methods for simulation and optimization of piecewise deterministic Markov processes

This book is focused on theoretical and numerical aspects of simulation and optimization for piecewise deterministic Markov processes (PDMP's). PDMP's have been introduced in the literature by M. Davis as a general class of stochastic hybrid models. They form a family of Markov processes involving deterministic motion punctuated by random jumps. The motion of a PDMP includes both continuous and discrete variables. The continuous state variable represents the physical parameters of the system under consideration. The discrete mode characterizes the regimes of operation of the physical system and/or the environment. The process is defined through three local characteristics, namely the flow describing the deterministic trajectory between two consecutive jumps, the intensity function giving the jump rate and the Markov kernel specifying the post-jump location. A suitable choice of the state space and these local characteristics provides stochastic models covering a large number of problems such as engineering systems, operation research, economics, management science, biology, internet traffic, networks and reliability. The class of PDMP's is thus considered and recognized as a powerful modeling tool for complex systems.

However, surprisingly few works are devoted to the development of numerical methods for PDMP's to solve problems of practical importance such as evaluation and optimization of functionals of the process. The main objective of this book consists in presenting mathematical tools recently developed by the authors to address such problems. This book is not only focused on theoretical aspects such as proof of convergence of the approximation procedures but is also concerned with its applicability to practical problems. The approach we are proposing is general enough to be applied to several application domains. In particular, our results are illustrated by examples from the field of reliability.

Our approximation technique is based on the discretization using quantization of the underlying discrete-time Markov chain given by the post-jump locations and jump times of the PDMP. This strategy enables us to address a large class of numerical problems. In particular, in this book we focus, on the one hand, on the computation of expectation of functionals of PDMP's with applications to the evaluation of service times. On the other hand, we are interested in solving optimal control problems with applications to maintenance optimization.